Systems Biology Lab
2024-11-01
Now discuss Sample space, Event space and probability of an Event for the Chocolate data.
All that needs to be proven concerning probabilities can be proven using these axioms and axioms of set theory.
Example: prove that \(P(A) + P(\overline{A}) = 1\)
\[ \begin{align*} A \cup \overline{A} = \Omega \quad &\Rightarrow P(A \cup \overline {A}) = P(\Omega) = 1 \\ A \cap \overline{A} = \varnothing \quad &\Rightarrow P(A \cup \overline{A}) = P(A) + P(\overline{A}) \\ &\Rightarrow P(A) + P(\overline{A}) = 1 \end{align*} \]
A conditional probability is calculated on a subset of the event space. For example on the subset \(B\).
Definition of the probability of Event \(A\) conditional on Event \(B\)
\[ P(A|B) = \frac{P(A \cap B)}{P(B)} \]
\(P(B)\) is called a normalization term: it re-scales \(P(A \cap B)\) to \([0,1]\)
Assume: codons have equal probability to be drawn; we make a random draw from the 64 codons
The product rule is derived from the definition of conditional probability
\[ P(A \cap B) = P(A|B) P(B) \]
In case of many Events \(A_1\), \(A_2\), …, \(A_n\) that form a partition set of \(\Omega\), we have
\[ P(B) = \sum_{i=1}^n P(B \cap A_i) = \sum_{i=1}^n P(B|A_i)P(A_i) \]
We discard the assumption of equal probability of codons.
Expressing \(P(A|B)\) in terms of \(P(B|A)\).
Since \(P(A \cap B) = P(A|B) P(B) = P(B|A) P(A)\)
We have
\[ P(A|B) = \frac{P(B|A)P(A)}{P(B)} \]
When given conditional probabilities on a partition set \(\mathcal{F}=\{A_1,\ldots,A_n\}\):
\[ P(A_j|B) = \frac{P(B|A_j)P(A_j)}{\sum_{i=1}^n P(B|A_i)P(A_i)} \]
freefreeCalculate the probability that an email is spam given that it contains the word free.
free in email), \(S\) (email is spam) and \(\overline{S}\) (email is not spam)Two Events \(A\) and \(B\) are independent when
\[ P(A|B) = P(A), \] otherwise they are dependent
In words, if \(P(A|B)=P(A)\) then:
This implies:
\[P(A \cap B) = P(A) P(B)\]
and
\[ P(B|A) = P(B) \]
A random variable is a function that maps every outcome in a Sample space to a number (usually from \(\mathbb{N}\), \(\mathbb{Z}\) or \(\mathbb{R}\))
Reason: use these numbers as input for probability distribution functions to enable the calculation of probabilities
Example: toss of a coin,
\[ \begin{align*} \text{tail} & \rightarrow 0 \\ \text{head} & \rightarrow 1 \end{align*} \]
Example: TIGRFAM protein classes \[ \begin{align*} \text{TIGR00001} & \rightarrow 0 \\ \text{TIGR00002} & \rightarrow 1 \\ \vdots & \rightarrow \vdots \\ \text{TIGR04571} & \rightarrow 4570 \end{align*} \]
Example: glucose concentration
\[ g\,\text{mM} \rightarrow g \]
A probability distribution (function) is a function that assigns a probability to every Event in an Event space
Example, \(k\) pips when throwing a die:
\[ p(k) = \frac{1}{6} \qquad k \in \{1,\ldots,6\} \]
Example, probability of head in 1 toss of a coin (tail \(\rightarrow k=0\), head \(\rightarrow k=1\)):
\[ p(k) = \theta^{k}(1-\theta)^{1-k} \qquad k \in \{0,1\} \] Example, number of heads \(k\) in \(n\) tosses of a coin:
\[ p(k) = {n \choose k} \theta^{k}(1-\theta)^{n-k} \qquad k \in \{0,\ldots,n\} \]
Example of a pdf:
The normal distribution \(f(x) = \frac{1}{\sqrt{2 \pi \sigma^2}} e^{-\frac{(x - \mu)^2}{2 \sigma^2}}\)
To calculate a probability using a pdf you have to integrate over an interval:
\[ P(X \in [a,b]) \equiv P(a \leq X \leq b) = \int_a^b f(x) dx \]
Important: a probability density is not a probability!
Joint probabilities are probabilities of Events that are Cartesian set products
Example: Members of parliament are Tory or Labour and Male or Female
\[ \begin{align*} &\Omega_\text{parliament} = \Omega_\text{sex} \times \Omega_\text{party membership} = \\ &\{\text{F}, \text{M}\} \times \{\text{T}, \text{L}\} = \{(\text{F},\text{T}),(\text{F},\text{L}),(\text{M},\text{T}),(\text{M},\text{L})\} \end{align*} \]
Joint probability distribution:
| sex | Tory | Labour |
|---|---|---|
| Female | 0.156 | 0.182 |
| Male | 0.477 | 0.185 |
Example of an Event: \(P(\{F\} \cap \{T\}) \equiv P(\{(F,T)\}) = 0.156\)
In the context of joint probabilities, a marginal probability is a result of applying the law of total probability
Example: \(P(\{F\}) = P(\{F\} \cap \{T\}) + P(\{F\} \cap \{L\}) = 0.156 + 0.182 = 0.338\)
\[ p(y\,|\,x) = \frac{p(x,y)}{p(x)} \]
Example: \(p(y\,|\,x=450)\)
The expected value of a random variable \(X\), written as \(\mathbb{E}[X]\) equals
\[ \mathbb{E}[X] = \sum_i x P(X = x) \]
Important: Differs from sample mean \(\frac{1}{n} \sum_{i=1}^n {X_i}\) of a set of samples \(X_i\) of \(X\).
The expected value of a function \(g(X)\) of \(X\), equals
\[ \mathbb{E}[g(X)] = \sum_i g(x) P(X = x) \]
\[ \mathbb{E}[a] = a \] in particular,
\[ \mathbb{E}[\mathbb{E}[X]] = \mathbb{E}[X] \]
\[ \begin{align*} \mathbb{E}[aX] &= a \mathbb{E}[X] \\ \mathbb{E}[X + Y] &= \mathbb{E}[X] + \mathbb{E}[Y] \end{align*} \]
The population variance of a random variable \(X\) is defined as
\[ \text{var}(X) = \mathbb{E} \left[ (X - \mathbb{E}[X])^2 \right] \]
Alternative (equivalent) definition:
\[ \text{var}(X) = \mathbb{E}[X^2] - \mathbb{E}[X]^2 \]
A set of independent samples \(X_i\) drawn from the distribution of a random variable \(X\) is called a set of Independent, Identically Distributed samples (i.i.d.) samples.
\[ \forall{i}: \quad \mathbb{E}[X_i] = \mathbb{E}[X] \]
For an unbiased estimator \(\hat{Z}\) of a random variable \(Z\) the following must hold:
\[ \mathbb{E}[\hat{Z}] = \mathbb{E}[Z] \]
Suppose \(X_i\) is a set of i.i.d. samples of \(X\)
The sample mean \(\overline{X} = \frac{1}{n}\sum_{i=1}^n{X_i}\) is an unbiased estimator of \(\mathbb{E}[X]\), i.e.
\[ \mathbb{E} [ \overline{X} ] = \mathbb{E}[\mathbb{E}[X]] = \mathbb{E}[X] \]
Proof: follows immediately from linearity of the Expectation operator.
The sample variance \(s^2(X_i) = \frac{1}{n-1}\sum\left( X_i - \overline{X}\right)^2\) is an unbiased estimator of \(\text{var}(X)\) , i.e.
\[ \mathbb{E}\left[ \frac{1}{n-1} \sum_{i=1}^n \left( X_i - \overline{X} \right)^2 \right] = \mathbb{E}[\text{var}(X)] = \mathbb{E} \left[ (X - \mathbb{E}[X])^2 \right] \]
Proof: see exercise in syllabus.
\[ P(S|F) = \frac{P(F|S) P(S)}{P(F|S)P(S) + P(F|\overline{S})P(\overline{S})} = \frac{\frac{1}{5}\frac{2}{10}}{\frac{1}{5}\frac{2}{10}+\frac{1}{100}\frac{9}{10}} = \frac{40}{49} \approx \frac{4}{5} \]